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Abstract 

A covering array t-CA{n,k,g), of size n, strength f, degree k, and order g, 
is a kxn array on g symbols such that every t xn sub-array contains every 
f X 1 column on g symbols at least once. Covering arrays have been studied 
for their applications to software testing, hardware testing, drug screening, 
and in areas where interactions of multiple parameters are to be tested. In 
this paper, we present an algebraic construction that improves many of the 
best known upper bounds on n for covering arrays 4-CA{n,k,g) with g = 3. 
The coverage measure IJ.t{A) of a testing array A is defined by the ratio be¬ 
tween the number of distinct f-tuples contained in the column vectors of A 
and the total number of f-tuples. A covering array is a testing array with full 
coverage. The covering arrays with budget constraints problem is the prob¬ 
lem of constructing a testing array of size at most n having largest possible 
coverage measure, given values of k,g and n. This paper presents several 
strength four testing arrays with high coverage. The construction here is a 
generalisation of the construction methods used by Chateauneuf, Colbourn 
and Kreher, and Meagher and Stevens. 


1 Introduction 

This article focuses on constructing new strength-four eovering arrays with g = 3 
and establishing improved bounds on the covering array numbers 4-CAN{k,3). 
This article also presents solution to the covering arrays with budget constraints 
problem by constructing many strength four testing arrays with high coverage. A 
covering array t-CA{n,k,g), of size n, strength t, degree k, and order g,is akxn 
array on g symbols such that every t xn sub-array contains every f x 1 column on 
g symbols at least once. It is desirable in most applications to minimise the size 
n of eovering arrays. The eovering array number t-CAN(k,g) is the smallest n for 
whieh a t-CA(n,k,g) exists. An obvious lower bound is 

g'<t-CANik,g). 
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In this paper, we describe a construction method which is an extension of the meth¬ 
ods developed by Chateauneuf, Colboum and Kreher (T] and Meagher and Stevens 
ca. This method improves some of the best known upper bounds for strength 
four covering arrays with g = 3. In the range of degrees considered in this paper, 
the best known results previously come from [J] ; in that paper, covering arrays are 
also found by using a group action on the symbols (the affine or Frobenius group), 
but no group action on the rows is employed. While for g = 3 the group that we 
employ on the symbols coincides with the affine group, we accelerafe and improve 
fhe search by also exploiting a group action on fhe rows as in |[I][T3l, and develop a 
search mefhod fhan can be applied effecfively whenever g > 3 and g — 1 is a prime 
power. 

There is a large liferafure lH |7]] on covering arrays, and fhe problem of deter¬ 
mining small covering arrays has been sfudied under many guises over fhe pasf 
fhirfy years. In f7], Hartman and Raskin discussed several generalizafions mofi- 
vafed by fheir applications in fhe realm of soffware fesfing. When testing a soff- 
ware sysfem wifh k paramefers, each of which musf be tested wifh g values, fhe 
fofal number of possible fesf cases is g^. For insfance, if fhere are 20 paramefers 
and fhree values for each paramefer fhen fhe number of inpuf combinations or fesf 
cases of fhis sysfem is 3^^ = 3486784401. A fundamenfal problem wifh soffware 
fesfing is fhaf fesfing under all combinations of inpufs is nof feasible, even wifh a 
simple producf ||9l|T0l. Soffware developers cannof fesf everyfhing, buf fhey can 
use combinatorial fesf design to identify fhe minimum number of fesfs needed to 
gef fhe coverage fhey wanf. The goal of mosf combinaforial fesfing research is to 
create fesf suites fhaf find a large percenfage of errors of a sysfem while having a 
small number of fesfs required. Covering arrays prove useful in locating a large 
percenfage of errors in soffware systems 0[l6l. The fesf cases are fhe columns of 
a covering array t-CA{n,k,g). This is one of fhe five nafural generalizafions in f?!. 
Covering arrays with budget constraints: A practical limifafion in fhe realm of fesf- 
ing is budgef. In mosf soffware developmenf environmenfs, time, compufing, and 
human resources needed to perform fhe fesfing of a componenf is sfricfly limited. 
To model fhis sifuafion, we consider fhe problem of creafing besf possible fesf suite 
(covering fhe maximum number of t-fuples) wifhin a fixed number of fesf cases. 
The coverage measure ft/(A) of a fesfing array A is defined by fhe rafio befween 
fhe number of disfincf f-fuples confained in fhe column vecfors of A and fhe fofal 
number of t-fuples given by Our objecfive is to consfrucf a fesfing array A 

of size af mosf n having largesf possible coverage measure, given fixed values of 
t,k,g and n. This problem is called covering arrays with budget constraints. 

We summarize fhe resulfs from group fheory fhaf we use. Lef be a Galois 
field GF(^) where q = p'” and p is prime. We adjoin to fhe symbol oo; it may 
be helpful to think of the resulting set 

A = F^U{oo} 

as the projective line consisting of ^ 1 points. Recall that the projective general 
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linear group of dimension 2 may be seen as the “fractional linear group”: 


PGL{2,q) = {a ■. X^X\xa = -where a,b,c,d G and ad — bc^ 0} 

in which we define ^ = ool=0, 1 —00 = 00 — l=oo and - = 1. It is known that 

\PGL{2,q) \ = = (^+ 1)^(^ “ 1) its action on U { 00 } is sharply 

3-transitive. For the undefined terms and more details see ifTSl Chapter 7]. 

Pair-wise or 2-way interaction testing and 3-way interaction testing are known 
to be effective for different types of software testing ||3l [HI [El. However, soft¬ 
ware failures may be caused by interactions of more than three parameters. A 
recent NIST study indicates that failures can be triggered by interactions up to 6 
parameters ifTOl . Here we consider the problem of 4-way interaction testing of the 
parameters. The construction given in this paper improves many of the current best 
known upper bounds on A-CAN{k,g) with g = 3 and 21 <k <14. This paper also 
presents several strength four testing arrays with high coverage measures. 

2 PGL Construction 

Let X = GF{g — 1) U { 00 } be the set of g symbols on which we are to construct a 
A-CA{n,k,g). We choose g so that g — 1 is a prime or prime power. 

2.1 Case 1: Two starter vectors 

Our construction involves selecting a group G and finding vectors m,v G X^, called 
sfarfer vecfors. We use fhe vectors to form akx2k matrix M. 
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Let G = PGL{2,g— 1). For each a G PGL(2,g— 1), let be the matrix formed 
by the action of a on the elements of M. The matrix obtained by developing M by 
G is the kx2k\G\ matrix = [M“ : a G G]. Let C be the kx g matrix that has a 
constant column with each entry equal to x, for each x GX. Vectors u, v G X* are 
said to be starter vectors for a 4-CA{n,k,g) if any 4 x 2k subarray of the matrix 
M has at least one representative from each non-constant orbit of PGL{2,g — 1) 
acting on 4-tuples from X. Under this group action, there are precisely g-|- 11 
orbits of 4-tuples. These g -|- 11 orbits are determined by the pattern of entries in 
their 4-tuples: 

1. {[a,a,a,a]^ : a GX} 
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2. {[a,a,a,bY : a,b € X,a ^ b} 

3. {[a,a,b,aY : a,b &X,a b} 

4. {[a,b,a,aj^ : a,b G X,a b} 

5. {lb,a,a,a]^ : a,b € X,a ^ b} 

6. {[a,a,b,b]^ : a,b ^ X,a ^ b} 

7. {la,b,a,b]^ : a,b € X,a ^ b} 

8. {[a,b,b,a]^ : a,b G X,a ^ b} 

9. {la,a,b,cj^ : a,b,c ^ X,a b c} 

10. {[b,a,a,cj^ : a,b,c € X,a b c} 

11. {la,b,a,cj^ : a,b,c ^ X,a b c} 

12. {[b,a,c,aj^ : a,b,c € X,a ^ b ^ c} 

13. {[a,b,c,aj^ : a,b,c € X,a ^ b ^ c} 

14. {[b,c,a,aj^ : a,b,c € X,a ^ b ^ c} 

15. g — 3 orbits of patterns with four distinct entries. The reason is this. There 
are g(g — l)(g — 2)(g — 3) 4-tuples with four distinct entries and each orbit 
contains g(g- l)(g-2) 4-tuples as |T’GL(2,g- 1)| = g(g- l)(g-2). 

If starter vectors m,v exist in X^ (with respect to the group G) then there exists a 
4-CA(2kg(g —l)(g — 2) +g,k,g). We give an example to explain the method. 

Example 1. Let g = 3, k = 30, X = GF{2) U {oo} and G = PGL{2,2). The action 
of G on 4-tuples from X has 14 orbits: 

Orb 1: [0000,oooooooo, 1111] 

Orb 2: [0001,000oo,oooooo0,ooooool, 1110, llloo] 

Orb 3: [loooooo^ 1000,0111,oo000,0oooooo,oolll] 

Orb 4: [0100,oo0oooo,0oo00,ooloooo, 1011, looll] 

Orb 5: [llool,ooooloOj0010,1101,00oo0,oooo0oo] 

Orb 6: [lloooo,ooooiij0011,1100,00oooo,oooo00] 

Orb 7: [oo0oo0,0101,oolool,0oo0oo, 1010, looloo] 

Orb 8: [ooHoo^ looooi^ 1001,0110,oo00oo,0oooo0] 
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14 

[1 

ooooO, lOOoo 

Olloo^ooOOl, 
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The following are starter veetors to eonstruet a 4-CA(363,30,3): 

u = (Oil OO 11 oooooo 001 oooooo loo 10 ooooQoo 1100 OO 01 ) 

V = (llooooOllOlOOOoolOloolooOooOOOOlOoooooo). 

We used eomputer seareh to find u and v. One ean eheek that on eaeh set of 4 rows 
of M there is a representative from eaeh orbit 2 — 14. Thus, 4-CAA/^(30,3) < 363. 

2.2 Choice of starter vectors u and v 

The problem is to find fwo veefors u,v G sueh fhaf on eaeh sef of 4 rows of M 
fhere is a represenfafive from eaeh orbif 2 — 15. To defermine whieh veefors work 
as sfarfers, we define fhe sefs d[x,y,7\ for posifive infegers x,y and z as follows: 

d[x,y,z] = {{ui,Ui+x,Ui+x+y,Ui+x+y+z) : 0 < / < k - 1}1J 

{{^i^^i+x-iVi^x+y^Vi^x+y+z) • 0 < / < k — 1} 

where fhe subseripfs are faken modulo k. For eompufafional eonvenienee, we par- 
fifion fhe eolleefion of ( 4 ) ehoiees of four disfinef rows from k rows info disjoinf 
equivalenee elasses. 

Formally, lefS'be fhe sef of all ( 4 ) 4-eombinalions of fhe sef {1,2, ...,k}. Define 
a binary relation /? on 5 by puffing 


{51,52A3A4}^ {/l, 4 , 4 ^ 4 } iff 

{^i +d,S 2 + d,S 3 +d,S 4 + d} = 14,4)4^4} for some r/ G N 

where all of fhe addition is modulo k. Beeause R is an equivalenee relation on 
S, S ean be partitioned info disjoin! equivalenee elasses. The equivalenee elass 
determined by { 51 , 52 ^ 3 ^ 4 } ^ S is given by 

[{ 51 , 52 ^ 3 ^ 4 }] = {{‘5'i +d,S2 + d,S3+d,S4 + d}\0 <d <k-\}. 

Wifhouf loss of generality, we may assume fhaf 0 = < 52 < < 54 for eaeh 

equivalenee elass represenfafive [{5i,3'2,3'3,3'4}]. As an illusfrafion, when X = 
{0,1,2, ...,7}. S is partitioned info 10 disjoin! equivalenee elasses: 

[{0,1,2,3}] [{0,1,2,4}] [{0,1,2,5}] [{0,1,2,6}] [{0,1,3,4}] 
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[{0,1,3,5}] [{0,1,3,6}] [{0,1,4,5}] [{0,1,4,6}] [{0,2,4,6}] 

A distance vector {x,y,z,w) is associated with every equivalence class [{^i ,^2 A 3 A 4 }] 
where x = S 2 — si, y = — S 2 , z = S 4 — 53 , w = si — S 4 mod k. The fourth distance 

is redundant because x + y + z + w = k. We rewrite the equivalence class of 4- 
combinations [{^'i, 52 , 53 , 54 }] as 


[x,y,z] = {i,i+x,i + x+y,i + x + y + z}\i = 0,1,2,...,^- 1} 


For k = 8 , [1,1,1] = [{0,1,2,3}], [1,1,2] = [{0,1,2,4}], [1,1,3] = [{0,1,2,5}], 
[1,1,4] = [{0,1,2,6}], [1,2,1] = [{0,1,3,4}], [1,2,2] = [{0,1,3,5}], [1,2,3] = 
[{0,1,3,6}], [1,3,1] = [{0,1,4,5}], [1,3,2] = [{0,1,4,6}], [2,2,2] = [{0,2,4,6}]. 

Lemma 1. Let S be the set of all 4-combinations o/{l,2,3, ...,fc}. Then S can be 
partitioned into disjoint equivalence classes 


[x,y,z] = {i,i+x,i + x + y,i + x + y + z}\i = 0,1,2,...,^- 1} 
where X = 1,2,..., y = x,x-\- \,...,k — 1 andz = x,x+ \,...,k— 1 such that 

(i) 2x4-y4-z < k 

(ii) when x = z, x<y < 

There are no further classes distinct from these. 

Before proving the result, we give an example. When S is the set of all 4- 
combinations of {0,1,2,3,4,5,6,7}, S can be partitioned into 10 disjoint classes: 
[1,1,1], [1,1,2], [1,1,3], [1,1,4], [1,2,1], [1,2,2], [1,3,1], [1,3,2] and [2,2,2]. 


Proof Let (x,y,z,w) be the distance vector corresponding to equivalence class 
[{3 'i, 3'2,3'3,54}]- Classes [{3 'i,3'2A3A4}], [x,y,z], [2>>va] and [w,x,y] are 

the same. Without loss of generality, we choose [x,y,z] as class representative if 
X <y, X < z. Thus 1 < v < ^, y = x,x4- \,...,k— 1 and z = v,v+ \,...,k— 1. We 
consider three cases: (i) x = w, (ii) x = z, (hi) x = y. If w = x, then the classes [x,y,z] 
and [x,x,y] obtained from distance vector (x,y,z,x) are the same equivalence class. 
The classes of the form [x,x,y] are generated under case (hi) as well. In order to 
avoid repetition, w has to be strictly greater than x. That is, w = k —x — y — z>x 
which implies 2x + y+ z < ^. If z =x, then the classes [x,y,z] and [x,w,x] are the 
same where y + w = ^ — 2x. Thus it is sufficient to consider the classes of the form 
[x,y,x] for y < j only. Hence the lemma follows. □ 

All the equivalence classes are enumerated by the following algorithm. 

Equivalence-Classes(A:) 

Input: k 

Output: All [x,y,z] classes. 

for 1 to I do 
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for X to ^ — 1 do 


ify> ^then 


for x+lto^ — 2x — y — 1 do 
add [x,y,z] 

end for 
else 

if y == and x == then 

add [ii?] 

else 

for z<— xto^ — 2x — y— 1 do 
add [x,y,z] 

end for 
end if 
end if 
end for 
end for 

Theorem 1. Let X = GF{g — 1) U {oo} and G = PGL{2,g— 1). If there exists a 
pair of vectors u,v G such that each d[x,y,z\ has a representative from each 
of the orbits 2 — 15, then there exists a 4-CA{2kg{g — l){g — 2) + g,k,g) covering 
array. 

Proof Let m,v G X* be veetors sueh that eaeh J[x,y,z] has a representation from 
eaeh of the orbits 2— 15. Using m, v, weereate the matrix [M^,C]. Let { 51,52 U3U4} 
be a member in S. By Lemma 1, there exists three positive integers xq, yo and zq 
sueh that { 51 , 52 , 53 , 54 } G [xo,yo,Zo]- h is given that d[xo,yo,Zo] has a representative 
from eaeh of the orbits 2-15. In other words, if we look at the rows 5i, 52 , 53 , 54 
of M, we see representative from eaeh of the g -|- 11 orbits. Consequently, beeause 
PGL{2,g— 1) is 3-transitive on X, [M^,C] is a 4-CA(2%(g — l)(g — 2) +g,k,g). 

□ 

At this stage, we make a few remarks about the size of equivalenee elasses 
defined by above ehoiees of x,y and z. 

1 . k^O mod 2 : 

If k is an odd integer, eaeh elass eontains exaetly k distinet ehoiees from 
the eolleetion of ( 4 ) ehoiees and henee there are Z = distinet 

elasses of size k. 

2 . k = Q mod 2 : 

If k is an even integer, | ean be written as sum of two positive integers a and 
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b where a<b'm different ways. 

Case i : If A: ^ 0 mod 4, a class of the form [a,^,a] contains only | distinct 
choices. There are total equivalence classes of the form [a,b,a\ with size 
I and the remaining classes are of size k. 

Case 2 '.If k = 0 mod 4, a class of the form [a,b,a\ contains only | distinct 
choices and a class of the form [a, a, a] where a = | contains only | distinct 
choices. Here we get total ^ — 1 equivalence classes of size | , exactly one 
class of size | and the remaining classes are of size k. 

For k = there are 10 equivalence classes. The classes [1,3,1] and [2,2,2] are 
of size 4 and 4 respectively and the remaining 8 classes are of size 8 each. Thus 
8x8 + 4 + 2= (^). 

2.3 Case 2: Two vectors u, v and a matrix Ci 

If we do not find vectors u and v such that each d[x,y,z] contains a representative 
from each of the orbits 2 — 15, we look for vectors that produce an array with 
maximum possible coverage. In order to complete the covering conditions, we add 
a small matrix Ci. We give an example below to illustrate the technique. 

Example 2. Let k = 21 and g = 3. Here we do not find vectors u and v such that 
each d[x,y, 7 \ contains a representative from each of the orbits 2 — 15. For k = 
21, there are 285 [x,y,z] classes. All classes [x,y,z] are obtained by the algorithm 
Equivalence-Classes. One can check that for the vectors 

M = 00001010oolooool0 oooo 001 OO 1 

V = OOOOlOOooOOoolOOOlool 1 loo 

there is a representative from each orbit 2 — 15 on 276 of the d[x,y, 7 \ classes. Table 
[T] shows nine classes which do not have representative from all the orbits: 

Table 1: List of classes not having representative from all the orbits 


Class 

Missing orbits 

d[l,2,2] 

10 

d[l,5,6] 

2 

r/[l,6,12] 

5 

r/[l,13,5] 

9 

d[2,3,8] 

6 

d[2,7,3] 

10 

^f[2,12,3] 

13 

r/[3,6,8] 

6 

r/[3,7,7] 

10 


In order to complete the covering conditions, we add a small matrix C\. 





Cl 


/ OO 0 1 1 

OO 1 1 oo 

0 1 OO 1 

0 oo 0 0 

10 0 0 

oo 0 0 oo 

oo oo oo 1 

0 1 oo 1 

0 0 10 
10 0 0 

oo oo oo oo 

oo 1 oo 1 

0 110 
0 oo 1 0 

0 0 0 oo 

oo 0 0 1 

oo 1 0 0 

1111 
0 oo oo 0 

0 oo oo oo 

\ oo 0 0 1 


0 oo oo oo 1 \ 

0 0 0 1 0 

1 0 1 oo 0 

1 0 0 0 0 

0 oo 1 oo 0 

0 oo oo oo 1 

0 0 0 1 0 

oo 0 1 1 0 

1 0 0 0 0 

1 oo 1 oo 0 

0 0 1 oo 0 

0 1 oo oo oo I 

oo 1 oo 1 0 

oo oo 0 0 0 

1 oo 1 0 0 

oo 0 0 0 oo 

0 1110 
0 0 10 0 
1 0 1 oo 1 

oo oo 1 0 0 

oo 0 oo oo \ j 


We use computer search to find matrix C\. This matrix has the property that every 
choice of four rows in [1,2,2], [2,7,3] and [3,7,7] contains at least one represen¬ 
tative from orbit 10; every choice of four rows in [2,3,8] and [3,6,8] contains at 
least one representative from orbit 6; each choice of four rows in [1,5,6], [1,6,12], 
[1,13,5] and [2,12,3] contains at least one representative from orbit 2, 5, 9 and 13 
respectively. We also need to use the following matrix 


C = 


/O 1 

0 1 


oo\ 


yO 1 


to ensure the coverage of all identical 4-tuples. Therefore, [M^, Cf, C] is a 4- 
CA(315,21,3). 


2.4 Case 3: One vector u and a matrix Ci 

For ^ = 37 to 58, we use one starter vector and a C\ matrix of order kxl with i<k. 
Tables|2j[3l|4]and[5]give a list of starter vectors and matrix Ci that improves the best 
known bounds. When the new bound is marked with an asterisk, post-optimization 
has been applied (see Section 
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3 Improving the solutions 


We examine two methods to obtain small improvements on the computational re¬ 
sults obtained. 

3.1 Extending a solution 

Until this point, starter vectors have been developed by applying a cyclic rotation 
of the starter vectors in addition to the action of PGL on the symbols. As in |[T3]I . 
one can also consider fixing one row, and developing the remaining k—\ cyclically. 
This can be viewed as first finding a solution of fhe fype already described onk — \ 
rows, buf requiring an additional properly. For fhe 4-subsels of {0,—2}, 
equivalence classes are defined as before, wilh arilhmefic modulo k—V. 

= {{s\+d,S2 + d,S'i+d,sn + d}\0 <d <k — 2}. 

For 3-subsels {fi, t 2 , ta } of {0,..., ^ — 2} we define furlher equivalence classes as 

1}] = {{^1 + dd2 + dd3 +d^k— 1}|0 <d < k — 2}. 

If we can place an enlry in posilion A: — 1 lo extend fhe lengfh of each sfarfer veclor 
so fhal every one of fhe (old and new) equivalence classes represenls each of fhe 
orbils 2 — 15, we oblain a 4-CA of degree k. 

The polenlial advanlage of Ibis approach is fhal a solulion for degree k—\ can 
somelimes be extended lo one of degree k wilhoul increasing fhe size of fhe cov¬ 
ering array produced. Indeed we found fhal fhe solutions for — 1 G {32,34,35} 
do ensure fhal fhe new equivalence classes also represenl each of fhe orbils 2 — 15. 
Hence we oblain fhe following improvemenls. Old indicales fhe bound oblained 
by applying our melhods lo t, Improved gives fhe bound by applying fhe melhod 
fo ^ — 1 and ensuring fhal fhe new equivalence classes represenl all orbils: 


k Old Improved 

k Old Improved 

k Old Improved 

33 399 387 

35 423 411 

36 435 423 


3.2 Randomized Post-optimization 

Nayeri, Colbourn, and Konjevod |[T4]| describe a posl-opfimizalion slrafegy which, 
when applied lo a covering array, exploils flexibilily of symbols in an allempl lo 
reduce ils size. We applied Iheir melhod lo fhe arrays provided here, and lo arrays 
oblained by removing one or more rows. Because Ihe melhod is described in delail 
elsewhere, we simply reporl improvemenls for eighl values of k. Basic gives Ihe 
bound from slarler vectors. Improved gives Ihe bound on A-CAN{k, 3) after posl- 
oplimizalion: 


k 

Basic 

Improved 

k 

Basic 

Improved 

k 

Basic 

Improved 

19 

309 

300 

20 

309 

303 

21 

309 

305 

22 

309 

307 

27 

351 

345 

28 

363 

360 

34 

411 

410 

37 

435 

433 





10 










4 Covering arrays with budget constraints problem 


In this section we present several strength four testing arrays with high coverage 
measure for g >3. The coverage measure ft 4 (A) of a strength four testing array 
A is defined by the ratio between the number of distinct 4-tuples contained in the 
column vectors of A and the total number of 4-tuples given by {\)g'^- Note that 
the coverage measure of a covering array is always one. For computational con¬ 
venience, we rewrite the coverage measure in terms of equivalence classes [v,y,z] 
and d[x,y,z] as follows: 


M4(A) 


^ |[x,y,z]| X number of distinct 4-tuples covered by d[x,y,z] 

x,y,z 




We search by computer to find vectors v wifh very high coverage measures. Tables 
12 and |7] show vecfors wifh high coverage, fhe number of fesf cases (n) generafed 
by our fechnique, and fhe besf known size wifh full coverage. Comparison of 
our consfrucfion wifh besf known covering array sizes shows fhaf our consfrucfion 
produces significanfly smaller testing arrays wifh very high coverage measures. 


5 Conclusions 

In fhis paper, we presenf a consfrucfion mefhod of sfrengfh four covering arrays 
wifh fhree symbols fhaf combines an algebraic technique wifh computer search. 
This mefhod improves fhe currenf besf known upper bounds on 4-CAN{k,g) for 
21 < k < 74 and g = 3. We have also proposed a consfrucfion of sfrengfh four cov¬ 
ering arrays wifh budgef consfrainfs. In order fo fesf soffware wifh 25 parameters 
each having fhree values, our consfrucfion can generate a fesf suite wifh 153 fesf 
cases fhaf ensure wifh probabilify 0.93 fhaf soffware failure cannof be caused due to 
inferacfions of fwo, fhree or four parameters whereas fhe besf known covering array 
in 14] requires 363 fesf cases for full coverage. The resulfs show fhaf fhe proposed 
mefhod could reduce fhe number of fesf cases significanfly while compromising 
only slighfly on fhe coverage. 
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Table 2: Improved strength four covering arrays for g = 3. 


Starter vectors and matrix Ci 


New 

bound 


Old 

bound 


21 


22 


27 


U = (OOOOlOlOooloooolO oooo OOlool) 

V = (0000100 CXD 00 CXD 10001 oo 111 oo^ 

/ oo oo 0 0 1 oo oo 0 0 1 oo oo 0 0 0 oo oo 10 0 oo^ 
0 1 looOOoolOOool looOOllooooO 
1 looOOOoooolOooool lOOOlooooO 
loolOOool lOOoolOOoolOlOool 
Cl=| 001 lOOOoollOOoooolcxiOOloocxj 
oo 0 0 0 0 0 0 oo 0 1 1 CXD oo 0 10 0 oo 0 

oo0101°o0101 looooOlOlll l°o 

oo \ oo 0 oo ®o 1 1 0 oo oo oo 100010 oo 0 oo 

VlOOOOlOOOOOooOOOooOOlOiy 


u = (000001 looOooOl lOoolooooooOloo) 

V = (OOOlOOlOoolooooOooOlcxD lOooool) 

/O oo oo 0 0 0 oo oo oo 0 0 0 oo oo 0 0 0 oo oo oo 0 0\ 

oo oo 0 0 0 oo oo oo 0 0 oo oo oo 0 0 0 oo oo oo 0 0 oo 

1 oo 1 oo 0 0 1 oo 1 oo 1 oo 1 oo 0 0 0 1 oo 1 0 

Ci=|01110011°o0ooll001100°oloo 

o 0 0 oo oo 1 0 1 oo 0 0 oo oo oo 0 0 1 oo 0 0 1 oo 

oOool 1 lOloolooOOl 11001 l°o0 
Voo Qooool OOooooooQooO 101 000 I J 


I ooooooOooOO®'^®'^ 1 OoO 1 1 0100 oo^ 

I ooooooOooOoo 01 oo 00001 oooooo^ 


305* 


315 


307* 


315 


M= (110101b 

V = (1 looOoolOl b 

^ 010101101010101010101010101 ^^ 
010101010101010110101010101 

\o°ooooooooooioooooooooooo°oooy 


Ci = 


345* 


378 
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Table 3: Improved strength four covering arrays for g = 3 (continued). 


Starter vectors and matrix Ci 


New 

bound 


Old 

bound 


28 


30 


32 


33 


34 


35 


35 


U = (looooOOoooolooOl 101 111 ooQoo OlOlooooool) 

v= ^ooioi lool lOooOOOooloooolOooooOooOOoo 01) 

OO 0 0 0 oo 0 0 0 00000®*^ 0 0 oo 00 oo0®o®o0®'^\ ^ 

oo 0 0 10 1 oo 0 1 0 oo oo 0 10 0 oo 0 1 0 1 oo 0 1 1 0 I 

1 0 0 0 1 ool 0*^ oil 

OooQooOOOOooOl 0 1 0°o01 0°o001 000000/ 


Cl = 


(Oil oo 11 ooooooQO 1 oooooo \ oo 10 ooooQoo 1100 oo 01) 

V = (1 looooOl 101000°ol01°ol°o0°o000010°o°o°o) 

u = (ool 100010°ol 1 Iooloo010°ooo0100°ooo0°o°o010) 

V = (oo000°ol°o°o0°o0001 10°o°o 100°o0°o 1 lool 1111) 

Obtained from CA(387,32,3) 

M= (00 OO 101 oooooo lOOlooOlOooooOooOooOlooooOool 1 1 1 1) 

v = (1100°oloo01°ol0110°o°o0°o°o01 Iool01001°o000) 

Obtained from CA (411,34,3) 

M = 01 ooQoooo 1000 oo 01 ooooQoo 1 oo 111 ooooooQ 1 oo 01000 ooj 

V = 0°o0011 loo0°ol 10°ol l°ol 10°o010010000°° l°o°o0 


360* 


363 


387 


387 


410* 


411 


423 


383 


393 


409 


417 


423 


429 


429 
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Table 4: Improved strength four covering arrays for g = 3 (continued). 


k 

Starter vectors and matrix Ci 

New 

Old 



bound 

bound 

36 

Obtained from CA (423,35,3) 

423 

441 

37 

u — (001°ol0°ol°o01000°ol 100°ol0111 1 oo001°o°o°ocxj00°o) 

Cl: 37 X 35 matrix 

433* 

441 

39 

U — (001°o°ol lool l°o0001°ol I°ol01oo°o°ol°o0°o0010°o00°o°o0) 

Cl: 39 X 34 matrix 

441 

453 


u = (ooOO 1 °°0 1 OooooQooQ 101111 ooooQ 1 1 CXJOO 10000°o0°ooo 1 QoaQoo 1) 



41 

Ci: 41 X 34 matrix 

453 

465 

42 

u = (ooOl 1 1 oo1cxjcxj100°o101°o01000°o 01 l°ol01001 loo00°oloooooo) 

Cl: 42 X 35 matrix 

465 

471 

46 

U = (oo00000°ol 100010°o 101 1 °o01 °o001 10°o°o°o°ol l°ol 101°ol01°o) 

Ci: 46 X 33 matrix 

477 

483 

47 

u = (ooOOl l°ol 101°ol°o000°oloo01°o00°ol 1 1010°o00°o°o°ol0oooolooooloooo) 
Ci: 47 X 33 matrix 

483 

489 

48 

U = (01°o°o°ol I°o01ool0101 1 I°o°o001°o°o°o0°ol 10010°o0°o°o000100°o00°o) 
Cl: 48 X 33 matrix 

489 

495 


u = (cxjOoooo 10101 l°o000°°°°l 1 oo1cxj1001°°°°°°°°°°1 1 



51 

cxiOoolooOl 111001001°o00) 

Cl : 51 X 32 matrix 

501 

507 
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Table 5: Improved strength four covering arrays for g = 3 (continued). 


k 

Starter vectors and matrix Ci 

New 

bound 

Old 

bound 

55 

U = (looooloolooOool 1 Iooooloo0010°o00°o001 101 loolooO 

00°ol 1 ooooO 101 °o001 1 10°o°o) 

Cl: 55 X 30 matrix 

513 

519 

57 

u = (oo 1 OooooooOO 1 Ioo01°ol0°ol 1001°ol°ooo001 looool 10 

110111010°o°ol°o0°o0000°o01) 

Ci: 57 X 29 matrix 

519 

531 

58 

u = (oo0oocxj00101°o0010°o0°o1°o1000°o0°o 1 1001°o00010°ol 11 

ooooool 101 101 looooOooOoo^ 

Ci: 58 X 29 matrix 

525 

531 

63 

U = (1 101ool0°ol00°ooooo00101°o°o0°o0°o°ol°o010°ol looooooOl 

10°ol01 10001°o0°ol 1 oocxj0oo0°o1 1) 

Ci: 63 x26 

537 

549 

67 

U = (010101°ol 100°ol00°ol looooooooOl 10°o01 1 1 loooolOl loo0°o 

1101 ooOooooOoo 101 °°°° 1 °°°° 10000°o00) 

Ci: 67x25 

555 

561 

70 

11 = (^\ ooOO 1 oo 1 1 oo 1 ooooooOoo 1 1 ooOooOoo 1 ooOOO 1 1 ooOoooooooo 111 
°o0101001°o01001 Ioooo010000°ol0°oool 100) 

Ci: 70x24 

567 

573 

72 

u = (oooo000°o 1010°o°o°o°o°o0101 1 1000°ol 101 looOl 1 101°o0°o°ol°o00 
ooloolooooOlO^^lOl 100°°01°°°®°®1°®°®0°®) 

Ci: 72x24 

573 

579 

74 

ll = (^\ OoOO 1 0°®°°0 1 OOOOOO 111 OOOO 1 ooooO 100°°°°°°®° 10°° 101 101 1°° 

001100001 °°°°0°°0°°0°°°° 101100°°1°°01°°111°°) 

Ci: 74x24 

585 

591 
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Table 6: A comparison of the number of test cases (n) produced by our con¬ 
struction with high coverage measure and best known n for full coverage. For 
g = 5, the elements of GF{4) are represented as 0,1? 2, and 3; here 2 stands for 
and 3 stands for x+l. _ 


(gF) 

Vector V with good coverage 

Our Results 
n (fi) 

Best known 

n O 

(3,16) 

00001001ooooO 1 1 oo 1 cx> 

99 (0.828) 

237 

(3,17) 

0000010ooc>o 101 ooO 1 oo 1 

105 (0.851) 

282 

(3,18) 

OOOlOooOcolOOlool 1 loooo 

111 (0.864) 

293 

(3,19) 

OOOOlOOlOooOlooOool 1 loo 

117 (0.883) 

305 

(3,20) 

00001 lOlOlc^OoolO'^ool loo 

123 (0.892) 

314 

(3,21) 

0000101Ooo 1 oooo 1 OooooOO 1 oo 1 

129 (0.906) 

315 

(3,22) 

0000011ooOooO 11 Ooo 1 ooooooO 1 oo 

135 (0.913) 

315 

(3,23) 

000000 1 oocoO 1 0 1 oo 1 Ooo 1 Ooooooo 1 

141 (0.923) 

315 

(3,24) 

0000000 1 ooooO 1 0 1 oo 1 Ooo 10 1 OOOO 1 

147 (0.924) 

315 

(3,25) 

000000001 looOooOl looOlooOool loo 

153 (0.930) 

363 

(3,28) 

1 OOOoOOoOOO 1 OoO 1101111 OoOooO 101 OOOOOO 1 

171 (0.957) 

383 

(3,29) 

01 OooOOoo 1 ooOoooooo 101 ooOOooOOO 1 1 1 oo 1 0 

177 (0.961) 

392 

(3,30) 

0 1 1 oo 1 1 OOOOOoOO 1 OOOOOO 1 oo 1 OoOOoOoO 1 1 OOooO 1 

163 (0.969) 

393 

(3,35) 

01 ooOoooo 1 OOOooO 1 ooooOoo 1 oo 111 ooooooO 1 ooO 1 OOOoo 1 

213 (0.979) 

429 

(3,36) 

1 looOl lOooooOOool 1110101 looOOl OOOOOOOOOO lOOooOoo 

219(0.981) 

441 

(3,38) 

1 oo 1 oo 1 1 1 ooooO 1 Ooo 1 OoOOoOOO 1 OoOOoOoOOOOO 1101 OOOO 1 OOoO 

231 (0.985) 

447 

(3,39) 

OOlooool lool looOOOlool loolOl OOOOOO looOooOOlOooOOooooO 

237 (0.986) 

453 

(3,40) 

1 OOooooOOOO 1 OOOO 1 oo 1 OooOOOooooooOoo 1 Ooooo 1 oo 1 ooO 111 ooO 1 

243 (0.988) 

465 

(4,18) 

ooo 1 002 1 oooooo2 1 020oo2 

436 (0.851) 

760 

(4,19) 

000012101 Ioo01oo0<x>221 

460 (0.866) 

760 

(4,20) 

0000112101202oo0221oo2 

484 (0.878) 

760 

(4,21) 

ooooo 1102101 0 oc 2 oo 022 1 oo 

508 (0.887) 

1012 

(4,22) 

0000001102oo02021oooo01ool 

532 (0.894) 

1012 

(4,23) 

ooooooo 1 2 1 02 1 0 °ooo 20 11 2 oo 1 

556 (0.898) 

1012 

(4,24) 

00000000121ooO 11 oo02oo0oo 112 

580 (0.899) 

1012 

(4,25) 

000000000121220°o011 °o2012oo 

604 (0.901) 

1012 

(4,26) 

00100<x>2221110102c<.0022oo020oo2 

628 (0.921) 

1012 

(4,27) 

01000 = 2221 110102oo0022oo020°=2 

652 (0.928) 

1012 

(4,28) 

01110o=0102oo0211 10022001=01001 

676 (0.933) 

1012 

(4,29) 

0=0=0122101 =000220200221220=02 

702 (0.937) 

1012 

(4,30) 

1 0'^20oo020oo2c>o2co0 1 oo2222oo022002oo 1 

726 (0.943) 

1012 
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Table 7: A comparison of the number of test cases (n) produced by our con 
struction with high coverage measure and best known n for full coverage (con 


tinned). 


{g,k) 

Vector V with good coverage 

Our Results 
n (^) 

Best known 

« 14| 

(5,21) 

1 10131300°o30010°°oo3203 

1265 (0.834) 

1865 

(5,22) 

3oo32011 200 oocx300oo0oo 10010 

1325 (0.842) 

1865 

(5,23) 

OOO 200 O 3 100oc203021332320 

1385 (0.854) 

1865 

(5,24) 

003°o21022212300032302310 

1445 (0.860) 

1865 

(5,25) 

oo200oo0oo'=o31020oo300303oooo33 

1505 (0.869) 

2485 

(5,26) 

202002211 OOOooO 121031OOOO2300 

1565 (0.873) 

2485 

(5,27) 

oooo03002030oo000oo 11 co0031301 oo3 

1625 (0.880) 

2485 

(5,28) 

013333130320°olool003200310300 

1685 (0.883) 

2485 

(5,29) 

00012212oo010<=o3110031020031010 

1745 (0.891) 

2485 

(5,30) 

33001 <x,0oo000330ooc>o010012oo1313001 

1805 (0.894) 

2485 

(5,31) 

033°o21333010313°o303320030012020 

1865 (0.895) 

2485 

(5,32) 

310031000°o330130321 <x.oo03031111310 

1925 (0.897) 

2485 

(5,33) 

ooOO 10c>ooo3oo0oo2oo01 coOOoo 1 2222ooooQ'^ooQ20oo 

1985 (0.904) 

2485 

(5,34) 

oooo3oo00101001ooOooOO 1 oo002oc01110231112 

2045 (0.906) 

2485 

(5,35) 

1203003303oc0oo013233310oo032020003220 

2105 (0.906) 

2485 

(5,36) 

12022oc3203230023223220001010200<x>2230 

2165 (0.912) 

2485 

(6,25) 

000403014003033404320°olo°°° 

3006 (0.811) 

6325 

(6,26) 

<>o0oo4002140401001301001 1444 

3126 (0.819) 

6456 

(6,27) 

433c>ooo01 c>oot20oo03020ooc<)0oo00401 

3246 (0.826) 

6606 

(6,28) 

4023031100232200°o21 oooo2020020 

3366 (0.829) 

6714 

(6,29) 

00oc400231033013434012303 34400 

3486 (0.834) 

6852 

(6,30) 

1 oooocx342oo4040004oo 104oo03034oooo0300 

3606 (0.836) 

6966 

(6,31) 

44122002oo2000020202031 oo42044001 

3726 (0.838) 

7092 

(6,32) 

44441341oo424000°°oo040004410103400 

3846 (0.846) 

7200 

(6,33) 

0330344c<.0232133100313000030°o4303c>o 

3966 (0.855) 

7320 
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